wisdomofthecrowdsfandomcom-20200214-history
WisdomOfTheCrowds Wiki
Wisdom Of The Crowds Wiki On this page you are introduced to a phenomenon called 'Wisdom of the Crowds', which refers to the finding that the aggregate of a set of proposed solutions from a group of individuals performs better than the majority of individual solutions. You will learn about the scientific background and the current discussions that evolve around this topic. This wiki was written by: Annemarie Buth, Nelleke van Schagen, Rolf van der Plas and Marieke Kil In light of the course Supercrunchers at Technical University Eindhoven. Summary Nowadays, people rely more and more on the internet for guidance in their decision making. Through online browsing people can find recommendations that they used to get from their social environment. We can now exhaust a gigantic pool of opinions rather than just a handful of friends. The internet is an ideal tool for this, because there are hardly any costs for a provider to keep track of the visitors’ behavior and to automatically aggregate and analyze this information. Also, the costs are negligible for doing this for ten site visitors versus (hundred) thousands It is found that the aggregated data from all site visitors can propose better, more tailored, solutions than each of these individual solutions could do individually. This finding is also referred to as ‘The Wisdom of the Crowds’ (wisdom of crowds). This heuristic finding is not only applicable in internet recommendations contexts, but in all problems concerning quantity estimations, general world knowledge, and spatial reasoning. Let’s come back to the use of wisdom of crowds for making more tailored recommendation to customers. These algorithms are not only capable of providing guidance from the behavior of the masses, they can also identify what ‘people like you’ enjoyed; making the recommendations even more specific. This collaborative filtering is a kind of tailored audience polling. It makes sense that people like you can make quite accurate guesses of what type of music or movies you will like. One might expect that these algorithms based on behavior of the masses would narrow the choices to a limited set of ‘bestsellers’. However, the opposite appears true: they actually diversify usage. Often the algorithms can recommend you music or movies that you have never heard of and would probably have never found by browsing on your own. This is called the “long tail” of the preference distribution (Figure 1). The wisdom of crowds is not only present in the online retailing market, but can also be recognized in a TV show like Who Wants to Be a Millionaire. It appears that asking the audience produces the right answer more than 90 percent of the time, while asking an individual for help only produces the right answer less than two-thirds of the time. Unfortunately, there is also a downside to exploiting the long tail; a social cost so to say. The more successful these personalized filters are, the more we are deprived of a common experience. These personalized recommendations can be seen as ‘filters’ that sort of censor the information we receive to that what fits our already prevailing views. Through these personalizations our views are only confirmed and never questioned. Main Questions There are two main questions discussed with respect to wisdom of the crowds in the book ‘Super Crunchers’ by Ian Ayres (2008): one is explicitly discussed and the other more implicitly. The explicit question being covered in the book is what the concept of wisdom of the crowds entails. The possibilities and also the applications of wisdom of the crowds that are currently in use are mentioned and discussed; providing insight in the relevance of the matter and the future opportunities of use. The question that is brought forward implicitly in the case of the wisdom of the crowds, is whether this phenomenon is a good or a bad thing. Indirectly, the author takes a positive stance towards the predictive power of the wisdom of the crowds, by providing several examples of cases in which collaborative filters lead to personal benefits of the user or to estimated predictions which are more accurate than the human expert. However, not only the benefits are mentioned in the case as the author also shortly discusses the downsides of the phenomenon. He mentions the social costs which are often accompanied by the long tail. Ayres discusses the deprivation of a common social experience, due to the success of personalized filters. Why Incentive for this case The book starts with the case of wisdom of the crowds. Ayres (2008) starts his book with this topic probably to get the reader into the topic. The phenomenon of wisdom of crowds is perfect for this as it provides for many recognizable examples in all of our lives. The main reason for Ayres to elaborate on the wisdom of the crowds phenomenon is thus mainly because it so apparent in our daily lives in the form of, for example, collaborative filters. People are not always aware of it and can therefore also not be expected to realize both positive and negative effects of these collaborative filters. There is not one specific case that triggered the discussion of wisdom of crowds by Ayres, but there are plenty of examples that one can find in its own environment. Moreover, the author is quite the advocate of following the wisdom of crowds more often. This is also argued for by Surowiecki (2005) in his book ‘The Wisdom of Crowds’. Surowiecki refers to sociologists Jack B. Soil and Richard Larrick that state we feel the need to “chase the expert”. However, this is a mistake; a costly one. As they put it: “we should stop hunting and ask the crowd (Surowiecki, 2005, p. XV introduction). Thus, another reason for addressing wisdom of crowds is making people more aware of it and trying to teach people how to benefit from this principle. Real life problem as a trigger In the introduction of the book of Surowiecki, he introduces the topic by describing the first known and documented research of the wisdom of the crowds principle. It is the year 1906 that a British scientist visits a country fair where local farmers and townspeople gathered to appraise the quality of each other’s cattle. Galton held the belief that only a very few people had the characteristics necessary to keep society healthy. During that day at the fair, he passed a weight-judging competition that considered the weight of an ox. Quite some people entered the competition varying from butchers and farmers to people that had no experience with cattle whatsoever. Galton was interested in figuring out what the “average voter” was capable of, because he ought them to be capable of very little. However, after averaging all individual guesses it turned that the average was a more accurate estimation of the ox’ weight than any of the individual estimations, including those of the so-called ‘experts’ had been. Although not a problem necessarily, this real life situation was the first trigger to research the existence of wisdom of crowds. The video below gives a brief summary of the experiment and how the phenomenon works. Also, the BBC puts the theory to the test by setting up their own little experiment. When considering the description of the first ‘experiment’, it could be argued that the wisdom of crowds concept was a coincidental finding. Actually, Galton had expected the opposite to be true: that the group was not capable of providing an appropriate average estimation. Although the results must have taken Galton by surprise, he did publish an article in Nature in 1907 (Galton, 1907) that describes the experiment and its findings. Unfortunately, he was not able to do much follow-up research as he passed away four years later. Reference Set To give a clear understanding of the wisdom of the crowds phenomenon, a set of six scientific papers is summarized. The first paper will elaborate on the scientific rationale behind the previously described example of Galton. The other papers are summarized to show new insights and applications of wisdom of crowds in the field of politics, crowdfunding, natural sciences and enterprises. The specific articles are selected for various reasons, which may include that they provide a deeper insight in the phenomenon of wisdom of crowds, show the newest applications or as some of these papers are seen as leading in their field of wisdom of crowds applications. 1. Are we Wise About the Wisdom of Crowds? The Use of Group Judgments in Belief Revision'' '' The first article shows that although in theory wisdom of the crowds should allow for better judgement making and that advice from groups should influence our judgement more than advice we get from individual people, we are reluctant accept this due to the fact that we are biased to weigh our own beliefs stronger than those of others’. 2. Wisdom of the Crowd within enterprises: Practices and challenges The second article shows how the wisdom of the crowd concept is currently applied in enterprises and identifies the issues that managers encounters. The article shows the impact of the concept and labels the current shortcomings for a better integration into enterprises. 3. (Wisdom of the Crowds) : 2010 UK Election Prediction with Social Media The third article is fundamental for understanding the influence and potential of the phenomenon for election outcomes. The paper shows that wisdom of crowds is a better indicator for election outcomes than currently used opinion polls, which could have substantial influence in daily life in times of elections. 4. Wisdom or Madness? Comparing Crowds with Expert Evaluation in Funding the Art The fourth article elaborates on another possible purpose of wisdom of crowds: crowdfunding. Funding decision-making processes are often led by the alternative method using field and/or innovation experts. The paper discusses whether we can use the wisdom of crowds algorithm to make better funding decisions for new ideas in a subjective environment. 5. Wisdom of crowds for robust gene network inference The fifth article is crucial for the understanding of the impact of wisdom of crowds on scientific findings. In a natural sciences study about genes, the application of wisdom of a crowds approach leads to better results than expert studies, indicating that the field of application is even wider than was previously thought. 6. Harnessing the Wisdom of Crowds in Wikipedia: Quality Through Coordination The sixth article presents the power of wisdom of the crowd: Wikipedia. The article identifies the factors that are crucial for the success of wikipedia and how the wisdom of the crowd influenced played his part in that. An overview of the scientific and non-scientific follow-up There are two main papers noteworthy concerning the phenomenon of wisdom of crowds. As mentioned previously, the first scientific appearance of the concept of wisdom of crowds was in 1907, when Galton published his article in the renowned scientific journal about his observations at the country fair. Since then, the article has been cited 670 times and has resulted in a large aftermath of publications about wisdom of crowds in various situations in a wide array of scientific fields. However, the largest share of papers about wisdom of crowds followed after the year 2004, the year in which James Surowiecki published his book "The Wisdom of Crowds". Surowiecki’s book made it to the bestseller list and popularized the phenomenon among the public, as it caused the larger audience to get to know about wisdom of crowds. A possible explanation for the large surge in popularity of the concept and the revival of the scientific interest after Surowiecki’s publication, is the rise of big data in modern society. The relevance of community knowledge is now more relevant than ever due to the increased connectivity enabled by modern technologies. The scientific interest in the topic has increased notably in the last 13 years, which is demonstrated by the high number of citations of the summary of Surowiecki’s book which can be found on Google Scholar. Moreover, entering “Wisdom of crowds” on Google Scholars provides a number of 161.000 results, illustrating that the scientific impact of the concept as introduced by Galton in 1907, and reintroduced by Surowiecki in 2004, is large. The first thirteen results on Google Scholar all have more than 100 citations, which is a relatively number compared to other scientific concepts. In terms of non-scientific literature, the impact of wisdom of crowds is also vast. There is a large amount of videos to be found online in which people perform their own experiments in testing whether wisdom of crowds actually works in practice. Interestingly, there are plenty of success stories proving that the theory works, creating a positive vicious circle of more people willing to try it for themselves out of curiosity. Furthermore, when entering the term wisdom of crowds on the Dutch media database LexisNexis, only in the Netherlands the concept is discussed in 361 articles in the period of 1980-currently. For a scientific term, this is a high number. More striking than that however, is the noticeable rise of times that the concept is mentioned in articles, indicating that it is becoming more and more a topical issue. Current scientific issues One of the main scientific questions related to this issue now, is how the growing availability of (big) data will unfold in the light of wisdom of crowds. How will the wisdom of crowds concept be used in the era where more and more data will be available, and how will the validity of the concept evolve? Another issue that is currently being discussed at MIT, is how to improve Wisdom of Crowds. In a lot of wisdom of crowd studies, all people are weighted equally, yet it is almost always the case that there are people that have more specialized knowledge. The researchers use a new technique which they call the “surprisingly popular” algorithm. This technique is supposed to extract the right answer in a more optimal way from large groups of people. This is done by instructing the people to answer what they think is the right answer, followed by what they think the popular opinion will be. In this way, it uses the knowledge of a well-informed subgroup within the larger crowd as a diagnostic tool that points to the right answer. For more details on how this exactly works one could look at the following article from Dizikes (2017) that describes the technique and provides some examples. What is most interesting about this article is that it shows that there are not only developments in the application of wisdom of crowds, but also in the technique behind it which is becoming more and more sophisticated. Common practice of super crunching The wisdom of the crowd method involves aggregation of individual estimations and taking the average, also known as taking the mean of a dataset. This is a very simple technique to give a rough estimation. It is well-accepted in the scientific environment, however, it is not a method that is used very regularly due to its simplicity. Many studies are required to apply much more complex statistical methods to give an accurate analysis. In practice, the phenomenon is not completely accepted and used to its full potential, because of the bias that we weigh our individual beliefs stronger than external advices. Next to that, trust issues come into play; people want to rely on the human expert instead of the data. A lot of people believe that the knowledge of an expert cannot simply be replaced by the aggregation of judgements of a group of people that have no specific knowledge on the issue concerned. Critique As indicated in previous paragraphs, there are four conditions for existence of wisdom of the crowds: diversity, independence, decentralization, and aggregation. Although this is mentioned quite regularly, authors might forget the effect of social influence on independency. Social influence, like group-thinking, conformity, and knowing the guesses of others, hinders independency and diversity, which could lead to irrational decisions and biases (Ball, 2014; Franch, 2013; Mollick & Nanda, 2015). This critique is especially relevant in the increasing amount of open and online communities, where social influence is likely the case. Engels (2017) argues that there is a way to restore these biases and wrong decision-making by the crowd. He explains the method of the ‘unexpected popular answer’, which means that the certainty of the answer given by each individual is taken into account. Another method to restore the probable social influence biases is by taking the confidence of an individual into account, thus instead of wisdom of the crowds use the wisdom of the confident (Madirolas & De Polavieja, 2014). More confident individuals, who gave a strong weight to their own estimate compared to the combined revealed group estimate and those who are thus less biased, are likely to give a much more accurate prediction. Another critique of this supercruncher example is the fact that wisdom of the crowds can lead to ‘the deprivation of a common experience’, as mentioned by Ayres (2008, p. 21). Moreover, this implies that personalization filters accomplished with wisdom of the crowds, might create a certain ‘bubble’ where information is shared between people with the same interests. The following YouTube video shows us that Facebook does have an algorithm that proposes you similar news items to the one you already liked, thus creating a unilateral news feed. Another example of this algorithm is the political bubble on Facebook. During the last Dutch political election, many Facebook users were confirmed in their own political opinions. For instance, if an user likes a certain news item about the mistakes of the political party VVD, Facebook will show many more similar items and creating a political ‘anti-VVD’ bubble for the user. This gives a highly unilateral news feed and does not show what is really happening in politics these days. The algorithm might do a good job at predicting what items we like, but do we really prefer this highly subjective news feed? Was it not the purpose of journalism to keep the people informed about the events and issues in the real world, instead of affirming people of their own prevailing beliefs? Moreover, there is also a philosophical issue regarding wisdom of the crowds that might easily be forgotten. When we are using wisdom of the crowds as a tool to live our lives (i.e. let the crowd determine what we read and do), we are not thinking, and thus missing out on our creativity, individuality and eventually our lives. Instead of developing our own interests and competences, the wisdom of the crowds shall do it for us. The consequences of this philosophical issue need more attention in future (scientific) research. Maybe even more important, we need to figure out whether this is what we want to achieve? Timeline Wisdom of crowds is a concept that was first applied in the early 20th century. The concept proves that the average estimation of a large group of people is highly accurate and in line with the true values of the estimated. The reluctant attitude of experts against the accuracy of the wisdom of the crowd has been heuristically overruled in the academic literature. Nowadays, enterprises are starting to implement the concept into their own companies. For example, the artificial intelligence software IBM Watson Health uses wisdom of the crowd to train its software to give recommendations about the diagnosis and/or treatment plan to the medical expert (IBM Watson Health, 2017). It is a matter of time and, more important, a matter of trust, that these recommendations change into the actual treatment plan. Moreover, this type of super crunching has also become common practice for many people who are unaware of the existence of the concept, as the application of the concept is redundant in daily life: e.g., using Wikipedia, Facebook, iTunes, Netflix, searches on Google, shopping at Amazon, hiking in mountains via paths, etc. The impact of the wisdom of crowds is therefore one of the most impactful concepts and almost fully accepted method (expert is still making a final call decision) of super crunching in the field and in society. However, as mentioned in the previous paragraph, there is some critique on the concept as well, as it can lead to deprivation of common experiences. As the amount of available data will keep growing in the coming years, this provides the challenge of how we as a society are going to handle these downsides. Because of that, wisdom of the crowds is going to become even more important in the era of big data ahead of us, that is one thing for sure. References Ayres, I. (2008). Super Crunchers: how anything can be predicted. Hachette UK. Ball, P. (2014). ‘Wisdom of the crowd’: The myths and realities. Bbc.com. Retrieved 5 May 2017, from http://www.bbc.com/future/story/20140708-when-crowd-wisdom-goes-wrong Dizikes, P. (2017). Better wisdom from crowds. MIT scholars produce new method of harvesting correct answers from groups. MIT News Office. Available at: http://news.mit.edu/2017/algorithm-better-wisdom-crowds-0125 7 May, 2017 Engels, J. (2017). Sleutelen aan de wijsheid van de massa. Trouw.nl. Retrieved 6 May 2017, from https://www.trouw.nl/home/sleutelen-aan-de-wijsheid-van-de-massa~ab9afa6f/ Franch, F. (2013). (Wisdom of the Crowds) 2: 2010 UK election prediction with social media. Journal of Information Technology & Politics, 10(1), 57-71. Galton, F. (1907). Vox populi (The wisdom of crowds). Nature, 75(7), 450-451. Gaissmaier, W., & Marewski, J. N. (2011). Forecasting elections with mere recognition from small, lousy samples: A comparison of collective recognition, wisdom of crowds, and representative polls. Judgment and Decision Making, 6(1), 73–88. Gloor, P. A., Krauss, J., Nann, S., Fischbach, K., & Schoder, D. (2009). Web science 2.0: Identifying trends through semantic social network analysis. Presented at IEEE Conference on Social Computing (SocialCom-09), Vancouver, British Columbia, Canada. IBM Watson Health. Cognitive Healthcare Solutions. (2017). IBM Watson Health. Retrieved 7 May 2017, from https://www.ibm.com/watson/health/ Madirolas, G., & De Polavieja, G. G. (2014). Wisdom of the confident: Using social interactions to eliminate the bias in wisdom of the crowds. In Proceedings of the Collective Intelligence Conference (pp. 10-12). Mannes, A. E. (2009). Are we wise about the wisdom of crowds? The use of group judgments in belief revision. Management Science, 55(8), 1267-1279. Category:Browse